Methods and apparatus for securely displaying digital images

ABSTRACT

The invention provides methods and apparatus for securing personalized or sensitive information appearing in digital images. Digital images containing information to be secured is processed and divided into a plurality of image cells. At least one image cell from the plurality of image cells is selected and redacted or altered to render the information to be secured with the selected image cells indecipherable. The invention also provides methods and apparatus for transmitting or streaming unredacted image cells alone or in combination with redacted image cells for secured viewing on a user browser or computer.

This application is a continuation application of U.S. patentapplication Ser. No. 11/769,674, filed on Jun. 27, 2007, which is acontinuation-in-part application of U.S. patent application Ser. No.11/612,425 filed on Dec. 18, 2006, now U.S. Pat. No. 8,612,854, whichclaims the benefit of priority to U.S. Provisional Patent ApplicationSer. No. 60/751,188 filed on Dec. 16, 2005, wherein each areincorporated herein by reference in their entirety.

FIELD OF THE INVENTION

The invention relates to securely displaying digital images of documentscontaining selected or confidential information. More particularly, theinvention relates to methods and apparatus for altering or redactingdigital images of documents for viewing such as personal checks in orderto secure sensitive information.

BACKGROUND

Converting a paper hardcopy document into a digital image may bedesirable due to the ease with which it can be transmitted and processedby a computer system. For example, images of documents can be scannedand transmitted via e-mail or shared on the Internet using a variety ofknown protocols such as HTTP or FTP. Obtaining a digital image of adocument may also be advantageous so that it can be manipulated oredited on a computer system. Electronic transmissions of digital imagesare frequently preferred over traditional postal methods due to speed ofdelivery and the ever increasing number of computer systems and ourreliance thereon.

The U.S. Congress recognized the advantages of using digital versions ofchecks over paper versions thereof by passing the Check 21 Act enactedin 2003. The Check 21 Act allows a financial institution to create adigital version of a processed check and make it accessible online tothe payer of the check, thereby eliminating the need for furtherhandling or return of the paper check. This computerized process greatlyreduces the time and costs associated with the processing of paperchecks, and hence enhances the efficiency of our banking system. It alsoprovides an easy and convenient alternative for bank customers tomonitor and manage banking related documents and activities. Onlinechecking offered by many financial institutions such as Bank of Americaand Wells Fargo allows customers to conveniently view digital images oftheir cashed checks.

However digital images of documents may often contain personal orsensitive information that a customer or a user may wish to conceal. Theterm “sensitive information” may include anything designated by a partythat should be secured for viewing online such as a customer's name,address or telephone number in conjunction with the customer's socialsecurity number, driver's license, or account number (FDIC FIL-27 2005).

At the same time, images must also reveal or contain sufficientinformation that is legible or usable to the user. For example, banksand other kinds of financial institutions allow a user to access anonline bank account and view a digital image of a processed check orfinancial document. But sensitive information such as the customer'sname, bank account number, routing number, address, telephone number,signature and other personal information, may appear on a computerscreen while viewing a digital version of the check. The security ofsuch personalized or sensitive information is compromised when it isdisplayed. The customer or user may therefore wish to hide or securesuch personalized information from computer hackers, identity thieves,or even from viewers with no ill intentions.

A variety of graphics editing programs are available today such as AdobePhotoshop that allows a digital image to be edited or altered manually.Original versions of an image may be created and saved as a new image.When using such software programs, a user often manually edits thepersonalized or sensitive content of the individual digital image withan editing tool such as a blur tool included with the software.Meanwhile, banks and other financial institutions usually process largequantities of checks and/or documents containing personalized orsensitive information. A financial institution may have numerouscustomers, each with an online bank account on which they can viewdigital versions of a processed or cashed check. The digital image maybe conveniently displayed to a customer while at the same time thefinancial institution can avoid having to mail or return the processedcheck to the customer. The ability to efficiently process a large numberof checks while protecting personal information contained thereinpresents significant challenges in online security and the prevention offraud or theft.

A need therefore exists to secure and protect personalized or sensitiveinformation within digital images of documents. It would be furtheradvantageous to protect available information online without having tomanually and singularly edit or alter the contents of the digitalimages.

SUMMARY OF THE INVENTION

The invention provides methods and apparatus for securing selectedinformation contained within digital images of documents. Variousaspects of the invention described herein may be applied to any of theparticular applications set forth below or for any other types ofinformation that is displayed for viewing. The invention may be appliedas a standalone tool or as part of an integrated software solutionagainst online fraud and identify theft. The invention can be optionallyintegrated into existing business processes seamlessly. It shall beunderstood that different aspects of the invention can be appreciatedindividually, collectively or in combination with each other.

A preferable embodiment of the invention provides image masking systemsand methods for providing online banking customers with necessary orshared information online without exposing sensitive data to potentialfraud. Such image masking can be performed real-time or on-the-fly to apresented image only without necessarily altering an originally scanneddocument. An originally scanned image can be remain intact andunaltered, in a preferable embodiment, while a new revised or redacteddigital image can be created. All of the image information for redacteddigital images provided herein may be transmitted for viewing, oralternatively, sanitized versions of a scanned document with just someof the image information may be sent instead to display non-sensitiveportions of an original document. The modified or redacted digitalimages provided in accordance with the invention offer customers accessto pictorial or graphical images of corresponding paper documentswithout compromising account or personal identity information thatshould be kept secured. Such information can be blurred by variousdigital editing techniques such as blurring or pixelating or shading, oralternatively, sections of the digital image containing sensitiveinformation may not be transmitted at all.

Accordingly, the invention can provide numerous advantages over otheravailable and complicated document security solutions by protectingsensitive customer data from fraud and identify theft without involvingcustomer education, adoption or enrollment. No enrollment process orchange of behavior is required on the part of customers. The securityprocesses provided in accordance with the invention can be performed onthe back end, invisible to users which can thereby enhance their senseof security without adding inconvenience or latency to the onlineexperience (automated backend security). While the masked images ofsecured documents herein may be helpful for daily use, customers canstill request to view and access an unredacted version of a documentwhen appropriate security measures are in place for additionalauthentication. Further password information may be requested, challengequestions may be presented calling for valid responses, one-time tokensor other strong authentication protocols may be employed.

Other goals and advantages of the invention will be further appreciatedand understood when considered in conjunction with the followingdescription and accompanying drawings. While the following descriptionmay contain specific details describing particular embodiments of theinvention, this should not be construed as limitations to the scope ofthe invention but rather as an exemplification of preferableembodiments. For each aspect of the invention, many variations arepossible as suggested herein that are known to those of ordinary skillin the art. A variety of changes and modifications can be made withinthe scope of the invention without departing from the spirit thereof.

INCORPORATION BY REFERENCE

All publications and patent applications mentioned in this specificationare herein incorporated by reference to the same extent as if eachindividual publication or patent application was specifically andindividually indicated to be incorporated by reference.

BRIEF DESCRIPTION OF THE DRAWINGS

The novel features of the invention are set forth with particularity inthe appended claims. A better understanding of the features andadvantages of the invention will be obtained by reference to thefollowing detailed description that sets forth illustrative embodiments,in which the principles of the invention are utilized:

FIG. 1 describes methods of securing personalized or sensitiveinformation displayed within a digital image of a document.

FIGS. 2-4 describe an embodiment of the invention that displays digitalimages of checks or other selected documents containing information tobe secured.

FIGS. 5-8 describe another embodiment of the invention that converts afirst digital image of a document such as a check into a second redacteddigital image containing selectively pixelated image cells that can besecurely displayed.

FIG. 9 illustrates a redacted digital image of a cashed personal checkthat is processed in accordance with another aspect of the invention.

FIG. 10 describes a system for securing digital images from alreadydigitally scanned documents or directly from paper documents with animage security module according to yet another aspect of the invention.

FIG. 11 describes a system for securing digital images with anelectronic image sanitizer that can filter out and send only sections ofan electronic image for secure viewing.

DETAILED DESCRIPTION OF THE INVENTION

The invention is directed to methods and apparatus for modifying oraltering digital images of documents in order to secure selectedinformation contained therein. The selected information may includepersonalized or sensitive information or any content which can beredacted or hidden from view in accordance with the invention. Theselected information may be hidden in an altered digital image so it canbe secured from view. Another aspect of the invention provides highthroughput automated processes and apparatus for securing or altering ofdigital images of hardcopies of documents. The invention furtherprovides computer systems and software programs that includeinstructions to carry out or perform various steps of such methods.

FIG. 1 describes an aspect of the invention that provides methods forsecuring selected information contained within a digital image. Thedigital image to be displayed may be derived from a scanned paperhardcopy document or any computer generated illustration that containspersonalized or sensitive information or content. Upon selection of thedigital image, it can be divided into a plurality of image cells thatcan be characterized or designated as either legible portions orillegible portions of the document or image. The legible portions of theimage may present non-sensitive or non-personal information that a usermay wish to display. The illegible portions of the image may howeverinclude sensitive, financial or personal information that should not bedisplayed. The image cells corresponding to illegible portions of theimage may be redacted to hide or obscure the underlying informationcontained in the image. Meanwhile, the image cells corresponding tolegible portions of the image can remain unredacted and visible so as toreveal content or images located therein. The resulting redacted oraltered digital image can be useful in many respects and communicatesufficient context or information but not so far as to compromise orunnecessarily reveal sensitive information. Accordingly, the redactedimage of the document can be rendered or available for display online ina controlled and secure manner.

A preferable embodiment of the invention provides a method of securelydisplaying digital images of banking related documents. The methodcomprises selecting a first digital image of a banking related documentcontaining personalized content, dividing the first digital image into aplurality of image cells, and then redacting at least one of the imagecells to provide a second digital image wherein the personalized contentin the banking related document is illegible (see FIGS. 2-4). Thismethodology can be applied to a variety of digital images correspondingto checks (personal or business check images) from financialinstitutions or any hardcopy document outside of banking transactions.Other preferable embodiments of the invention can be directed todocuments or digital images thereof such as deposit slips, bankstatements, brokerage statements, legal documents, credit card bills, aswell as tax documents or returns, driver's licenses, medical records orany other document containing personalized or sensitive information thata user may wish to hide or conceal from view on a computer or online. Itshall be understood that the personalized or sensitive information neednot be in the form of text, but may be rather a graphical image such asan illustration of an individual, fingerprint or biometric information.The documents secured in accordance with this aspect of the inventioncan originally exist as a paper hardcopy that can be scanned to createdigital images, or the documents may be stored as digital images incomputer readable memory such as a computer hard drives, flash memorydrives or other memory media.

A digital image of a check or a selected document can be chosen forsecure display in accordance with the invention as shown in FIG. 2. Thecheck or selected document can be designed with various shapes and sizesincluding relevant information printed thereon. Selected kinds ofpersonalized information may be displayed or appear within the samegeneral location or region on the face of the check or document. Forexample, the name and address information of an individual, e.g., payor,can be printed generally on the upper left hand corner 210 of the checkor selected document. The date appears often in the upper right hadcorner 220. The central portion 230 of the check or selected documentmay contain other kinds of sensitive information. For example, withrespect to any selected document, this location may include informationsuch as credit card numbers, bank account/routing numbers, accountstatements from financial institutions or graphical content. It shall beunderstood that the invention can be applied to protecting other kindsof non-textual information such as photographs, images, blueprints orschematics that should not be readily viewable by an observer. Withrespect to a check, as shown in FIG. 2, the central portion 230 oftenincludes the name of the payee or to whom the check is written. Thedollar amount for the check appears immediately below usually written orspelled out (one hundred dollars) as well as appearing numerically($100). A memo line is also provided o the lower left hand corner 240 ofthe check that identifies for what the check was written. A space orline 250 can be provided at the lower right hand corner of the checkwhere the signature of the payor appears.

As shown in FIG. 3, the check can be conceptually divided into aplurality of image cells in accordance with the invention. The digitalimage of the check can be divided into a grid consisting of five (5)columns×three (3) rows. As with other embodiments of the inventiondescribed elsewhere herein, the image cells within the grid may bedesignated or separated into either redacted or un-redacted portions orcells. In some embodiments of the invention, the number of image cellswithin the grid may be predetermined or fixed by the party who wants tosecure confidential information such as a payor or a bank having theaccount from which the check will be drawn against. When more imagecells are selected, greater control can be provided over which regionsof a check or a selected document can be secured in accordance with theinvention. It shall be understood that the grids described herein mayinclude any number of desired columns/rows and are not limited torectangular or square shapes and can include image cells with differentsymmetrical or asymmetrical shapes and sizes.

Alternatively, the number and/or kind of image cells can be userdefined. A user may select various image cell inputs designating thesize, shape, and number of rows and columns for a desired grid. Forexample, a digital image can be divided into a plurality of image cellslocated within a rectangular grid based on user input values with apredefined number of rows and columns, e.g., 4 columns×3 rows. The gridmay be characterized as a conceptual overlay upon the image so that theimage can be broken-up or segmented into image cells that pertain toboth redacted and un-redacted portions of the digital image. It shall beunderstood that image cells are not necessarily laid out in a grid likepattern with both rows and columns and can be alternatively arranged inany manner including a layout with cells that are aligned in onlyhorizontal rows or only vertical columns.

By selectively altering redacted portions of the digital image as shownin FIG. 4, sensitive information contained within the check or selecteddocument can be protected. For example, one or more selected image cellscontaining sensitive information can be identified within the grid.These image cells can be designated as the ones for editing orredaction. The image fragments contained within these selected imagecells or redacted cells can be altered in appearance in accordance withthe invention. Meanwhile, the remaining or non-selected image cells canbe left untouched so that corresponding image fragments containedtherein are plainly visible. Any graphics editing software tool orprogram may be used to alter or change the appearance of the imagefragments within the selected image cells so that the sensitiveinformation contained within the selected image cells will not belegible to an ordinary observer. In a preferable embodiment, theselected image cells can be grayed out (colored grey) entirely as shownso that any traces of the image fragments are not visible. Two selectedimage cells located at the upper left hand corner of the digital image,and two selected image cells located at the lower right hand corner, canbe redacted to protect sensitive information otherwise visible, namelythe name and address of the payor and a corresponding signature. Inanother preferable embodiment of the invention, a group of one or moreselected image cells may be deleted or omitted from a digital imagebefore delivering or transmitting it for secure viewing. The entireredacted digital image(s) to be viewed may thus comprise the entiregroup or all image cells (redacted and un-redacted) or only a selectedsubgroup of image cells (un-redacted) corresponding to the document(s).Accordingly, the resulting digital image of the check or selecteddocument can be displayed online or on a computer screen securelywithout sharing or disclosing sensitive information.

There are a variety of ways in which digital images can be redacted inaccordance with the invention. For example, the step of redactingselected portions of a digital image can be achieved by a combination ofone or more different means including graying or blacking out theseareas. For purposes of this invention, the term “redacted” can mean ablurred, obscured, removed, blocked out, or crossed out so thatunderlying information cannot be seen and is not apparent on its face toan ordinary observer. Preferably, personalized information protected inaccordance with the invention includes confidential subject matter suchas personal data, financial or other types of information that can orshould be shielded from general public viewing. Other ways to redactdigital images in accordance with the invention may include variouscombinations of one or more techniques such as masking in solid colors(e.g., black, grey, white), pixelating or pixelizing, encrypting and/orotherwise rendering selected regions of a digital image illegible orindecipherable. In preferable embodiments of the invention, some legibleportions of the digital image remain unredacted or plainly visible toreveal image fragments or contents of the document residing within theseportions. It is often useful to provide at least some context and torelay some minimal amount of information to an observer but not to theextent such that sensitive information is revealed beyond an intendedpurpose.

FIG. 5 illustrates another embodiment of the invention that securelydisplays a digital image of a personal check containing a variety ofsensitive information including a series of important numbers relatingto a banking account. For example, a 9-digit number American BankingAssociation (ABA) routing transit number can be found at the bottom ofthe check adjacent to an account number from a financial institution.The ABA number or routing transit number is a sensitive piece ofinformation that is often necessary to reorder checks, to set up directdeposits and recurring payments, or when preparing a wire transfer. Thecustomer bank account number is another piece of sensitive informationthat should be guarded carefully to prevent identity theft and fraud.For many online banking purposes, it is not necessary to display theseseries of numbers. For example, a bank customer may just want todetermine whether a check has been properly endorsed, or by whom, byviewing an image of the back of a check. The customer may also want toinspect the front face of the check to confirm there has been no forgeryor unauthorized alterations made. The unsecured display of suchinformation online thus presents exposure to both the customer and thefinancial institution with no intended benefit in such circumstances.Accordingly, any digital images of checks that are available online canbe secured in accordance with the invention to prevent unauthorizedviewing by hackers or other persons who gain access to viewable onlineaccount information.

As shown in FIG. 5, the check includes additional personalized orsensitive information such as the payor name and address, a signatureline, and routing transit/bank account numbers. Upon writing and cashingthe check, a digital image of the instrument can be created usingavailable scanning equipment and processes by a financial institution.It can be redacted and secured in accordance with the invention asdescribed elsewhere herein. However a financial institution such as abank may determine a methodology or process beforehand in which thecheck and other similar checks can be secured for online display. Forexample, the check can be initially scanned and saved. Digital imagescan be saved and loaded into memory of any type of computer system ornetwork maintained by the financial institution. The digital image maybe saved as a file that can be edited in accordance with the invention,or in other embodiments, the image may be derived from a stream of dataas with other embodiments described herein. It shall be understood againthat invention can be applied to electronic or digital images of anytype of document where it may be desirable to alter the content of theimage such that at least a portion of the image cannot be seen or isillegible. Electronic images of other documents may be selected besidespersonal checks such as deposit slips, bank statements, credit cardbills, tax returns, or any other documents that may contain sensitive orpersonal information. In addition, the digital image of a hardcopy of adocument may exist in various known formats including without limitationthe following commonly used graphics file formats for displaying digitalimages on the World Wide Web: JPEG/JPG, GIF, PNG, TIF, TIFF, BMP, PSD,WMF, EMF, PCX, PIC and PDF.

In a preferable embodiment of the invention, as shown in FIG. 6, thefinancial institution can divide the digital image of the check into aplurality of image cells. A grid can be formed over the check or itsimage to provide a series of image cells arranged in rows and columns.While any number of image cells may be chosen and arranged in differentways, the check as shown is divided into thirty-nine (39) sections. Thegrid can be thus designed with three (3) rows and thirteen (13) columns.When more image cells are defined in accordance with this aspect of theinvention, more flexibility is often provided in redacting selectedportions of the check. The number of selected image cells or the numberof rows or columns into which the digital image is divided may be basedon user input. In this embodiment of the invention, the grid is createdwith vertical and horizontal lines such that the image is divided intorectangular-shaped image cells while in other embodiments the grid maycomprise curved lines, or lines that do not form a regular pattern.Other embodiments may include any combination of the above, or mayinclude any other division of the image as may be desired to carry outthe invention. The image may be divided into any number of image cellswhich may be of any size, shape, orientation, or other configuration.

In some embodiments of the invention, a group of image cells formedwithin a grid can be identified by a marker to distinguish the imagecell relative to others within the entire electronic image. The markermay be a reference number according to a numbering convention. In anexemplary numbering system, the image cells may be designated withsequential numbers starting with zero (0) or one (1) in the upper leftcorner of the grid. The image cells may be assigned numbers from left toright (each column) and continue from top to bottom (each row). Whileany unique identifier or number may be assigned in any order, methodsare provided herein whereby each image cell is assigned a unique numberuntil all image cells are identified. Other numbering or marking systemsmay be used to designate the position of the image cells within the gridincluding but not limited to an alphabetical system (A-Z), analphanumeric system (A1, A2 . . . Z10) or a color coded system may beused to uniquely label the plurality of image cells. In some embodimentsof the invention, a simple binary-type cell identifier system can bealso adopted. Each image cell would not have to be uniquely identifiedfrom every other image cell. For example, each image cell correspondingto a portion of a digital image may be designated either as a redactedcell or not (un-redacted cell). Any or all image cells designated asredacted cells can be altered while un-redacted cells remain visible inaccordance with this embodiment the invention.

FIG. 7 illustrates a preferable embodiment of the invention whereby aplurality of image cells is identified using numerical cell identifiersranging from zero (0) to thirty-eight (38). The cell identifiers beginwith the number zero (0) in the upper left cell and may continuesequentially across the columns and in a row-by-row manner until allimage cells are assigned a number ending with the number thirty-eight(38). Three rows of thirteen image cells are therefore created toprovide a thirteen (13) by three (3) grid corresponding to portions ofthe digital image. In accordance with this aspect of the invention, thedigital image may be redacted according to selected cell identifiers.This redaction may be accomplished by selecting a group of one or morecell identifiers corresponding to portions of the digital imagecontaining sensitive information. The image fragments residing withinthese image cells can be redacted. Selecting cell identifiers forredaction may be accomplished by user input indicating which image cellidentifiers, and thus their corresponding image cells, are to beredacted. A computer program may also include instructions or computercode to automatically select which image cell(s) are to be redactedbased on corresponding image cell identifiers. Such computer code mayfurther carry out steps to automatically redact a digital imageaccording to selected cell identifiers. Accordingly, a first digitalimage can be altered by having at least one image cell redacted toprovide a second digital image wherein the redacted image cell isillegible, thus securing personalized content.

As shown in FIG. 8, a predefined group of image cells within the gridcan be selected for redaction. Because the name and address of anindividual often appears on the upper left hand corner of a check, imagecells 1 through 5 can be selected for redaction as these cellscorrespond to that portion of the document or digital image. At the sametime, since the routing transit and bank account numbers often appearalong the bottom edge of a check, image cells 28 through 38 can beselected for redaction as these cells correspond to that portion of thecheck. Any number of image cells may be selected for redaction in orderto partially or entirely secure information. For example, partialrouting transit numbers may partially appear while the entire accountnumber for a payor is completely illegible. In other instances it may benecessary or desired to reveal certain kinds of information appearing onthe check such as the fractional routing number (FRN). So the imagecells corresponding to this portion of the image (e.g., image cells 9through 10) are not selected for redaction. Upon selection of the imagecells that are to be redacted, a graphics editing tool or program can beused to redact those portions of the image in order to protect otherwisevisible information. In this illustrated embodiment of the invention,the image cells chosen for redaction were masked or rendered illegibleby pixelating the image cells or the portions of the digital imagecorresponding to the image selected cells. Other methods may be used toredact the image cells besides pixelating including but not limited tograying out, encrypting, blurring, masking, or drawing a line throughall or just a portion of an image cell. In alternative embodiments, theredacted cells can be blurred or pixelated, wherein the blurring orpixelating is achieved by utilizing a blur size parameter to designatethe size of the blurred area. A blur size may be, e.g., but not limitedto, ten (10) by ten (10) pixels.

The selected image cells to be redacted can be identified according tomarkers or cell identifiers that direct the editing tool or program asto which portions of the digital image should be redacted. It shall beunderstood that information to be secured within checks or any otherselected document in accordance with the invention may reside inslightly or entirely different locations therein. An appropriate numberof image cells may be chosen in order to offer at least some minimallevel of security to at least partially redact information positionedwithin a digital image since checks or any other selected document mayvary in size and shape. Accordingly, a financial institution such as abank may therefore decide beforehand how much or which portions ofcustomer checks should be generally available for online display.

FIG. 9 illustrates yet another embodiment of the invention that isapplicable to personal checks. The personal check in this figure appearsas it would to an ordinary observer on a display or monitor after it hasbeen partially redacted in accordance with the invention. In thisinstance, a grid with three (3) rows by four (4) columns was selectedthat divides the digital image into twelve (12) image cells. After thecheck is cashed, the payor/account holder/customer may want to view thecheck on a home computer through an online banking system. Through a webbrowser, for example, the payor or any other person may select forviewing the cashed check (front/back images). Regardless of whether theimage of the check has been intercepted or viewed by someone withoutpermission, the relatively sensitive information contained thereon issecured in accordance with the invention. The digital image of thealready scanned paper (hardcopy) document is redacted to concealinformation such as the payor name and address, the routing transit andbank account numbers, plus the payor's signature which could otherwisebe studied to commit forgery. Meanwhile, certain kinds of informationcan be visible to give some context and to convey some basic informationthat may be deemed useful but relatively non-sensitive or non-personal,e.g., date, name of payee, name of bank, dollar amount, purpose ofcheck. In this illustration, image cells 0 through 1 and 9 through 11were selected for redaction. The secured information has been redactedby pixelation and masked from view. The image cells could have beenalternatively redacted by other masking techniques described elsewhereherein, or otherwise blackened or grayed out, partially or completely.As illustrated in this embodiment, selected image cells can be partiallyrather than entirely redacted. This degree of controlled redaction canbe achieved based on the technique selected such as pixelation in thisembodiment. It may be desirable to pixelate only certain portions of theimage cells leaving other pixels corresponding to some image fragmentswithin such cells intact and viewable. As with other embodiments of theinvention, it may be thus desirable to only partially redact an imagecell containing personalized information.

While masked or redacted images provided herein may be advantageous forsecuring personalized content, a user or customer may wish to view theoriginal unredacted image or document. An alternative embodiment of theinvention may offer this by presenting a complete digital image to anauthorized person following an authentication procedure. A user may viewan original unredacted image online after providing for example a useridentification and password. It shall be understood that the inventionmay be also applied to digital image encryption/decryption schemes suchas those disclosed in U.S. Pat. No. 6,954,532 (Handley et al), which isincorporated by reference in its entirety herein. But preferableembodiments of the invention herein can provide document securitywithout encryption/decryption schemes or segmenting digital images intoconceptual layers. For example, redacted portions of documents (e.g.,pixelated) provided in accordance with the invention may be permanentlyaltered and considered unredactable (e.g., unpixelated) by anyoneincluding even the creator of the document. Other alternativeembodiments of the invention however provide temporarily redacteddocuments that can be unredacted, preferably without public/private keyencryption and decryption techniques wherein redacted images sharedonline include embedded public key information. Such keyless redactionand unredaction techniques such as pixelating/unpixelating according tographical editing software programs (e.g., Adobe PhotoShop) that areknown only by authorized users or viewers can be also be applied toembodiments of the invention herein. Alternatively, the unredacted imagemay be presented following some predetermined level of second levelauthentication online (or ordered through paper mail or made availableand a local branch of a financial institution).

In accordance with yet another aspect of the invention, as shown in FIG.10, one or more hardcopy paper documents can be modified to secureselected information contained therein. The digital image of an alreadyscanned document can be redacted, or alternatively, paper documents canbe scanned and converted directly into secured digital images. Forexample, an original unredacted group of one or more documents can bescanned to generate electronic images using image scanning tools orequipment. Such apparatus may have Optical Character Recognition (OCR)capabilities and be able to scan paper copies to generate unredacteddigital images according to a preselected digital image format such as aJPEG format, for example. The image(s) may be subsequently redacted andconverted by upon execution of a computer software program for redactingdigital images that resides in the memory of a computer provided inaccordance with another aspect of the invention described herein. Acomputer running the software program (redactor) may contain an imagesecurity module that redacts selected portions of the already scanneddigital images to secure certain information as described elsewhereherein. The image security module may optionally convert the firstunredacted digital image existing in a first format (JPEG) into a secondredacted digital image existing in a second format (PDF). It shall beunderstood that the redactor and/or image security module may operateindependently as a stand alone application software program or an add-ontool for commercially available image scanning and conversion programs.

FIG. 10 illustrates another embodiment of the invention whereby paperdocuments are directly scanned and secured by the image security module.It shall be understood that the image security module may be a computerprogram or set of instructions to carry out a method of redactingdigital images of hardcopy documents, or it may be a discrete set ofinstructions or a combination of subroutines to perform functions suchas the scanning and/or redacting of documents as described herein. Forexample, a group of one or more selected paper hardcopy documents can beprocessed by the image security module to redact certain portionsthereof. When the documents are relatively uniform in appearance, e.g.,shape, size, font, a common group of image cells corresponding tosimilar portions of the documents can be scanned and redacted inaccordance with other aspects of the invention described elsewhereherein. However when the documents to be scanned and redacted are notuniform, or if different regions of similar or uniform documents are tobe redacted, then the image security module can selectively redactdigital images accordingly such that resulting digital images of thedocuments protect information residing at different portions of thescanned documents. Either the redacted digital images of the paperdocuments can be rendered for immediate display, or they can be storedfor later viewing after being processed by the image security module tomask or hide sensitive information appearing on the face of thedocuments. Accordingly, paper documents can be scanned and directlyconverted into redacted digital images by the image security module.

Various aspects of the invention herein may scan paper documents, orconvert digital images of documents, into any digital image format. Thefollowing is a description of some of the most commonly used graphicsfile formats for putting graphics on the World Wide Web that may beapplied to the invention:

JPEG/JPG. Short for Joint Photographic Experts Group, the original nameof the committee that wrote the standard. JPG is an image file formatsupported on the Web that is a lossy compression technique designed tocompress color and grayscale continuous-tone images. The informationthat is discarded in the compression is information that the human eyecannot detect. JPG images can support 16 million colors and are suitablefor photographs and complex graphics. A user may have to compromise oneither the quality of the image or the size of the file. JPG may notwork well on line drawings, lettering or simple graphics.

GIF. Short for Graphics Interchange Format, another of the graphicsformats supported by the Web. Unlike JPG, the GIF format is a losslesscompression technique and it can support 256 colors. GIF may be viewedas a preferred format over JPG for images with only a few distinctcolors, such as line drawings, black and white images and small textthat is only a few pixels high. With an animation editor, GIF images canbe put together for animated images. GIF also supports transparency,where the background color can be set to transparent in order to let thecolor on the underlying Web page to show through.

PNG. Short for Portable Network Graphics, it is another graphicsstandard found on the Web but is not supported by all browsers. An imagein a lossless PNG file can be 5%-25% more compressed than a GIF file ofthe same image. PNG builds on the idea of transparency in GIF images andallows the control of the degree of transparency, known as opacity. PNGdoes not support animation like GIF does.

It shall be understood that the invention herein can redact and convertor produce document images from many other different types of imageformats such as TIF, TIFF, BMP, PSD, WMF, EMF, PCX, PIC and PDF formats.

Another aspect of the invention provides automated high throughputprocesses wherein a large number of digital images can be generatedand/or secured on-the-fly (real time). For example, many checks fromvarious financial institutions often adopt a relatively standard orsubstantially similar format. Because checks usually include the samekinds of information, their corresponding images can be similarlyredacted (batch redacted) without individual manual editing or redactingwith software programs such as Adobe PhotoShop. A variety of automatedprocesses and systems are provided herein to redact sensitiveinformation from digital images of documents for display online orthrough other communication channels as part of a high throughputprocess. With respect to checks, the image masking systems and processesherein can leverage the fact that personal or sensitive informationoften appears in common locations as described elsewhere herein. Thefront side of a check, for example, may include the payor address in anupper left side area while the routing number and account number mayappear along the bottom near the signature of the payor. Digital imagesof a plurality of checks, for example, can be secured or altered in anautomated batch process, thus requiring less time than would be requiredif the images were scanned and/or secured individually. A gridconstructed with a plurality of image cells can be designed for thechecks as described herein so that commonly selected (or the same) imagecells can be redacted on-the-fly for each corresponding check. Theselection of a sufficient number and/or location of image cells forredaction may be considered in view of modest variations existing asbetween different checks (different sizes, fonts and layouts). Someinformation on checks may be more edited or redacted compared to othersbut enough cells should be appropriately selected for redaction in orderto sufficiently mask the check images to afford at least some level ofprotection, e.g., at least 5 digits from a 9 digit routing transitnumber should be redacted while 6, 7 or more digits may be redacted insome checks. The digital images of the checks or any other documentcontaining information to be protected herein can be thereforedynamically divided and masked in accordance with the inventionon-the-fly, and preferably as part of a batch processing of checks orother documents by a financial institution, merchants and other partiesdesiring secure digital images.

For example, referring to FIG. 10, an image security module may performa batch security process to securely display cashed checks for viewingby banking customers. In a preferable embodiment of the invention, thehardcopy paper versions of the checks (presumably cashed and held by abank) can be digitally scanned using high volume paper scanners tocapture and create digital images of (both front and back sides) thechecks. A series of one or more image files such as PDF files can becreated containing electronic images of the checks in unredacted form.This image data can be thereafter processed by an image security moduleto create redacted electronic images of the checks for secure viewing inaccordance with other aspects of the invention described elsewhereherein. The image masking systems and methods herein can perform awrapper function around existing image content by taking an alreadyscanned document and blurring or redacting relevant parts of the digitalimage before presenting it to an end user. Alternatively, the digitalimage information corresponding to the checks upon scanning can be feddirectly into the image security module whereby the function ofredacting selected portions of the digital images are performed withoutgenerating unredacted digital versions of the checks. For someapplications, it may be more useful to only create and/or store redactedelectronic images of checks. No duplicate images are generated at allwhich reveal sensitive information according to this embodiment of theinvention. Accordingly, this aspect of the invention allows a pluralityof checks or any other documents with similarly located information tobe redacted more efficiently and faster in time than if the digitalimages were redacted individually.

Other embodiments relating to this aspect of the invention can providebatch processing of digital images for other kinds of documentscontaining confidential or sensitive information described elsewhereherein to protect against identity theft, fraud and other kinds ofillegal activity.

In an online application of the invention that securely displays digitalimages, the following data flow can be implemented: a user requests apage with a dynamic image content (such as a check); a server returns aHTML page with a reference to a back end service that can provide theimage; a browser intercepts the <IMG> tag and follows the SRC link toretrieve the image; the service behind the <IMG> link interprets therequest and locates the image in a repository (e.g., scanned documentscreated as shown in FIG. 10); and an image stream can be buffered andstreamed back to the client browser. Accordingly, a straightforward andsimple integration of the invention can be accomplished with an existingprocess whereby a server retrieves or receives an already scanneddigital image, which is passed through an image security module, e.g.,including ImageMask module, before sending the redacted (masked) imageto a client (customer browser).

A preferable embodiment of the invention performs a wrapper functionaround an existing image content call, for example, when a customerdesires to view a cashed check. This Java API function call can take theimage and mask relevant parts in accordance with the invention hereinbefore resolving an updated redacted image to an end user. The typicaldelivery process and image can retain intact, processing time can benegligible and no duplicate images are generated.

In accordance with another aspect of the invention, a database orrepository of digital images can be created as shown in FIG. 11. Itshall be understood that the repository may be maintained and integratedas part of an overall banking system or other network providing secureddisplay of sensitive documents. The plurality of digital images mayinclude both redacted and unredacted electronic images of documents.Such documents include bank checks, statements and any other documentscontaining sensitive information including those described elsewhereherein. The documents in the repository may be processed by an imagesecurity system (ISS) before their images are transmitted or streamed toviewers across a network like the Internet for viewing on a computer ordevice browser. The image security system may include one or moreservers and computers containing computer programs for securing digitalor electronic images of documents in accordance with other aspects ofthe invention. A server may retrieve a digital image from a repository,which is then passed through the image security system, before sendingthe redacted image to a client (customer browser). In addition to orinstead of containing an image security module (see FIG. 10) and otherprograms, a computer memory in the image security system may alsoinclude an electronic image sanitizer. The electronic image sanitizermay be a software program or module with computer implementedinstructions or code that filters out selected portions of an imagebefore transmission for viewing. The selected portions of the image maybe created and divided into multiple image cells as described elsewhereherein.

For example, the electronic image data for a document such as a cashedpersonal check may be stored in the repository 110. When the documentexists in its unredacted form, its image may be divided into a pluralityof image cells so that certain selected cells containing sensitiveinformation can be identified in accordance with other aspects of theinvention (e.g., FIG. 9). The image data corresponding to the selectedcells can be filtered out by the electronic image sanitizer. Thefiltered data can be discarded or optionally retained (XXXX) by thesecurity system. So only some and not all image cells of the documentare transmitted for secured viewing. The image data and related pixelinformation corresponding to areas of the document containing sensitiveinformation can be thus omitted from transmission. A sanitized document112 can therefore be made available for viewing on a computer browser114 that visibly excludes sensitive information.

Alternatively, a document may exist in the repository in its redactedform 116 following image data processing in accordance with otheraspects of the invention. In this embodiment of the invention, thedocument may have been already redacted by the image security module sothat sensitive portions of the document are not legible when rendered ona computer browser or device screen 119. Here the image security systemmay transmit directly or indirectly a complete set of image data for adocument 117 including both redacted and uredacted portions. Thetransmitted image data may include information corresponding to redactedimage cells (solid masked and/or pixelated image portions) andunredacted image cells. But in alternative embodiments of the invention,the data for some image cells (cross-hatch/shaded section), preferablythe ones corresponding to confidential or sensitive information in adocument, are not transmitted at all and omitted from the relayed imagedata to render a sanitized document 118 for viewing on a display. Thismay reduce the amount of data sent over the networks to a remote userdevice, and also provides added security in that the data correspondingto sensitive information does not even leave a secure computer networksuch as those operated by financial institutions. Selected image data ordata streams corresponding to repository documents can be buffered andstreamed back to a client browser or display for secure viewing.Accordingly, this aspect of the invention provides systems and processesfor sanitizing a digital representation of a document for viewing.

The embodiments of the invention which perform sanitization of documentsand data herein may be characterized as systems and processes forremoving sensitive information from a document or other medium so thatit may be securely distributed. When dealing with sensitive orclassified information, sanitization may convert an otherwise classifieddocument into an unclassified document. For example, a page of aclassified document may be sanitized for public release in accordancewith the invention. Classified information may be removed so that onlythe unclassified information is available or visible. A printed documentwhich contains classified or sensitive information will frequentlycontain significant information which is less sensitive. And, there maybe a need to release the less sensitive portions to uncleared personnelor members of the general public such as requests under the Freedom ofInformation Act (FOIA). The printed document may thus be sanitized toremove the sensitive information, or as with any of the embodiments ofthe invention herein, the document may be redacted to obscure theinformation instead. The terms sanitization and redaction as used hereinmay be applied to printed or paper documents as well as computer media,information and data as well.

It should be understood from the foregoing that, while particularimplementations have been illustrated and described, variousmodifications can be made thereto and are contemplated herein. It isalso not intended that the invention be limited by the specific examplesprovided within the specification. While the invention has beendescribed with reference to the aforementioned specification, thedescriptions and illustrations of the preferable embodiments herein arenot meant to be construed in a limiting sense. Furthermore, it shall beunderstood that all aspects of the invention are not limited to thespecific depictions, configurations or relative proportions set forthherein which depend upon a variety of conditions and variables. Variousmodifications in form and detail of the embodiments of the inventionwill be apparent to a person skilled in the art. It is thereforecontemplated that the invention shall also cover any such modifications,variations and equivalents.

What is claimed is:
 1. A method for automatically redacting a digitalimage, the method comprising: accessing a digital image for processingby an image security system, the digital image including sensitiveinformation and associated with a predetermined document type of aplurality of predetermined document types; accessing a cell divisionpattern associated with the predetermined document type; dividing thedigital image into a plurality of image cells according to thepredetermined cell division pattern; and determining at least one imagecell of the plurality of image cells that includes the sensitiveinformation; determining within the at least one image cell, a firstportion of the at least one image cell that includes the sensitiveinformation and a second portion of the at least one image cell thatdoes not include the sensitive information; and editing the digitalimage to redact the first portion of the at least one image cell andleaving the second portion of the at least one image cell unredacted. 2.The method of claim 1, wherein the plurality of predetermined documenttypes comprises at least one of: a check, a deposit slip, a bankstatement, a credit card bill, a tax return, a driver's license, amedical record, or a variation in font or layout of a document type inthe plurality predetermined document types.
 3. The method of claim 1,wherein editing digital image comprises at least one of: masking insolid color, pixelating, encrypting, or otherwise rendering the firstportion of the at least one image cell indecipherable.
 4. The method ofclaim 1, wherein the cell division pattern comprises a grid of imagecells of the plurality of image cells, wherein the grid comprises atleast one row and at least one column.
 5. An image security system forredacting a digital image, the image security system comprising: anon-transitory data store configured to store a plurality of digitalimages and cell division patterns associated with a plurality ofpredetermined document type; a computer processor programmed to: access,from the non-transitory data store, a digital image of the plurality ofdigital images, the digital image including sensitive information andassociated with a predetermined document type of the plurality ofpredetermined document types; access, from the non-transitory datastore, a cell division pattern of the plurality of cell divisionpatterns associated with the predetermined document type; divide thedigital image into a plurality of image cells according to thepredetermined cell division pattern; determine at least one image cellof the plurality of image cells that includes the sensitive information;determine within the at least one image cell, a first portion of the atleast one image cell that includes the sensitive information and asecond portion of the at least one image cell that does not include thesensitive information; and edit the digital image to redact the firstportion of the at least one image cell and leaving the second portion ofthe at least one image cell unredacted.
 6. The image security system ofclaim 5, wherein the plurality of predetermined document types comprisesat least one of: a check, a deposit slip, a bank statement, a creditcard bill, a tax return, a driver's license, a medical record, or avariation in font or layout of a document type in the pluralitypredetermined document types.
 7. The image security system of claim 5,wherein to edit the digital image, the computer processor is programmedto perform at least one of: masking in solid color, pixelating,encrypting, or otherwise rendering the first portion of the at least oneimage cell indecipherable.
 8. The image security system of claim 5,wherein the cell division pattern comprises a grid of image cells of theplurality of image cells, wherein the grid comprises at least one rowand at least one column.
 9. A non-transitory computer readable mediumcontaining program instructions for automatically redacting a digitalimage which contains sensitive information, wherein the programinstructions causing an image security system to: access a digital imagefor processing by an image security system, the digital image includingsensitive information and associated with a predetermined document typeof a plurality of predetermined document types; access a cell divisionpattern associated with the predetermined document type; divide thedigital image into a plurality of image cells according to thepredetermined cell division pattern; determine at least one image cellof the plurality of image cells that includes the sensitive information;determine within the at least one image cell, a first portion of the atleast one image cell that includes the sensitive information and asecond portion of the at least one image cell that does not include thesensitive information; and edit the digital image to redact the firstportion of the at least one image cell and leaving the second portion ofthe at least one image cell unredacted.
 10. The non-transitory computerreadable medium of claim 9, wherein the plurality of predetermineddocument types comprises at least one of: a check, a deposit slip, abank statement, a credit card bill, a tax return, a driver's license, amedical record, or a variation in font or layout of a document type inthe plurality predetermined document types.
 11. The non-transitorycomputer readable medium of claim 9, wherein to edit the digital image,the program instructions causes the image security system to perform atleast one of: masking in solid color, pixelating, encrypting, orotherwise rendering the first portion of the at least one image cellindecipherable.
 12. The non-transitory computer readable medium of claim9, wherein the cell division pattern comprises a grid of, image cells ofthe plurality of image cells, wherein the grid comprises at least onerow and at least one column.